The  Wellcome  Trust 

helping  medical  science  to  flourish 


PRISM 

UNIT  FOR  POLICY  RESEARCH 
IN  SCIENCE  AND  MEDICINE 


The  Wellcome  Trust  is  an  independent  charity  and  the  largest  non- 
government funder  of  biomedical  research  in  the  UK.  Within  the  Trust, 
the  Unit  for  Policy  Research  in  Science  and  Medicine  (PRISM)  is 
responsible  for  providing  research  and  advice  to  assist  senior 
management  in  making  decisions  on  issues  of  policy  by: 

•  evaluating  research  outcomes; 

•  auditing  scientific  activity  in  different  research  fields  and  countries; 

•  applying  novel  approaches  to  strategic  planning  and  priority  setting. 

As  well  as  carrying  out  policy  research,  PRISM  offers  two  unique  services 
to  funding  organizations,  policy  makers,  government  departments, 
universities  and  industrialists: 

•  SPIN  (Science  Policy  Information  News)  -  a  weekly  round-up  of  news 
in  biomedical  science  policy. 

•  ROD  (Research  Outputs  Database)  -  developed  by  PRISM  to  track 
research  outputs  in  biomedical  sciences.  For  the  first  time,  research- 
funding  agencies  are  able  to  identify  and  acquire  details  of  research 
papers  attributable  to  them. 


Women  and  Peer  Review 


An  audit  of  the  Wellcome  Trust's 
decision-making  on  grants 


LIBRARY 


•Gi«ja«ml  Coflactkms 


4% 


PRISM  Report  No.  8 


ACKNOWLEDGEMENTS 


The  study  was  carried  out  by  Dr  Jonathan  Grant  and  Lawrence  Low 
under  the  direction  of  Dr  Joe  Anderson  and  Dr  Suzanne  King. 
Magdalena  Vinter  and  Robert  Cottrell  assisted  with  data  collection.  Dr 
Grant  Lewison  provided  additional  advice  on  the  bibliometric  analyses. 
We  are  grateful  to  all  the  individuals  and  organizations  who  assisted 
with  the  study. 


2 


Women  and  Peer  Review:  an  audit  of  the  wellcome  trust's  decision-making  on  grants 


CONTENTS 


ACKNOWLEDGEMENTS 
EXECUTIVE  SUMMARY 

1  INTRODUCTION 

2  THE  OUTCOME  OF  WELLCOME  TRUST  GRANT  APPLICATIONS 

2.1  Project  grants,  1996 

2.2  Programme  grants,  1994-1996 

2.3  Senior  Research  Fellowships  in  Basic  Biomedical  Science,  1994/95-1996/97 

3  DISCUSSION 

3.1  Age  and  sex  characteristics  of  the  biomedical  community 

3.2  Conclusion  and  policy  implications 

REFERENCES 


2 

4 

5 

6 
6 
8 

10 

13 
13 
16 

17 


ANNEXES 

1  Bibliometric  analysis:  Methods 

2  Bibliometric  analysis:  Results 


18 
19 


Women  and  Peer  Review:  an  audit  of  the  wellcome  trust's  decision-making  on  Grants 


3 


A  recent  study  of  peer-review  scores  for  postdoctoral  fellowships  at  the 
Swedish  MRC  demonstrated  that  women  had  to  be  2.5  times  more 
productive  than  their  male  colleagues  to  get  the  same  peer-review  rating 
for  scientific  competence. 


We  have  audited  the  Wellcome  Trust  s  decision-making  on  grants  and 
demonstrated  that  there  is  no  evidence  of  sex  discrimination  in  the 
awarding  of  project  grants,  programme  grants  or  Senior  Research 
Fellowships  in  Basic  Biomedical  Science  (SBBF): 

•  Award  rates  are  about  the  same  for  men  and  women; 

•  Publication  records  of  successful  applicants  are  also  similar. 

There  is  evidence  that  women  do  not  apply  to  the  Trust  for  project  or 
programme  grants  in  the  proportions  that  would  be  expected  from  the 
number  of  female  academics  working  in  UK  universities.  This  is  not  the 
case  for  SBBF. 


It  is  recommended  that  funding  bodies  should  work  together  to 
identify  the  reasons  why  women  do  not  apply  for  grants  in  the  numbers 
expected. 
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INTRODUCTION 


Over  the  past  five  years  the  issue  of  women  in 
science  has  captured  the  policy  agenda. 
Concern  about  the  failure  of  women  to 
advance  to  more  senior  grades  in  academia  led 
to  the  establishment  of  a  Committee  on 
Women  in  Science  and  Engineering  in  1993 
which,  in  its  report,  The  Rising  Tide,  made  a 
number  of  recommendations  aimed  at  increas- 
ing the  number  of  women  in  science.1  This 
was  the  impetus  for  an  attitudes  survey,  pub- 
lished by  the  Wellcome  Trust's  Unit  for  Policy 
Research  in  Science  and  Medicine  (PRISM), 
which  highlighted  factors  that  may  be  dissuad- 
ing women  from  entering  a  scientific  career.2 
More  recently,  a  high-profile  analysis  of  the 
peer-review  system  of  the  Swedish  Medical 
Research  Council  (MRC)  demonstrated  that 
women  suffer  discrimination  because  of  their 
sex.  The  authors  proved  that  female  applicants 
for  Swedish  MRC  postdoctoral  fellowships  had 
to  be  2.5  times  more  productive  than  their 
male  colleagues  to  get  the  same  peer-review 
rating  for  scientific  competence.3 

Given  the  results  of  the  Swedish  study,  the 
objective  of  the  current  report  is  to  examine 
whether  there  is  inadvertent  sex  discrimination 
in  the  Trusts  grant-giving  practices  by  focusing 
on  two  questions: 


1  Are  applications  from  women  more  likely  to 
fail  than  from  men? 

By  examining  successful  and  failed  grant 
applications,  we  investigated  whether 
applications  to  the  Trust  from  women  are 
more,  or  less,  likely  to  fail  than 
applications  from  men. 

2  Do  women  need  to  have  a  better  publication 
record  than  men  to  win  grants? 

We  tested  the  hypothesis  that  women 
need  to  have  a  more  impressive  track 
record  than  men  to  be  awarded  grants  by 
looking  at  the  publication  records  of 
women  whose  applications  are  successful, 
and  comparing  them  with  men. 

To  answer  these  questions  we  examined 
applications  to  the  Trust  for  project  grants, 
programme  grants  and  Senior  Research 
Fellowships  in  Basic  Biomedical  Science 
(SBBF).  These  schemes  were  chosen  as  they 
represent  the  broad  range  of  support  provided 
by  the  Trust,  and  are  aimed  at  scientists  at 
different  stages  in  their  careers. 


Women  and  Peer  Review:  an  audit  of  the  wellcome  trust's  decision-making  on  grants 


5 


THE  OUTCOME  OF  WELLCOME  TRUST  GRANT  APPLICATIONS 


2.1  PROJECT  GRANTS,  1996 


Project  grants  account  for  33  per  cent  of  all 
awards  and  27  per  cent  of  expenditure  by  the 
Trust  in  1995/96. a  Grants  are  normally  made 
to  holders  of  established  posts  in  a  university  or 
institution  (not  to  staff  of,  or  fellows  funded 
by,  Research  Councils)  and  typically  provide 
support  for  up  to  three  years.4 

All  project  grant  applications  that  were 
received  by  the  Trust  in  1996  were  examined. 
Data  on  the  sex,b  age  and  past  three  academic 
addresses  of  successful  and  unsuccessful  app- 
lications were  recorded.  Table  1  describes  the 
outcomes  of  the  1387  applications  on  which 
information  was  collated.0  The  overall  success 
rate  is  27.5  per  cent  which  is  similar  for 
women  (26.9  per  cent)  and  men  (27.5  per 
cent).  An  alternative  way  of  looking  at  Table  1 
is  to  compare  the  proportion  of  female  applic- 
ants with  the  proportion  of  women  who  suc- 
ceeded and  failed  in  their  applications.  For 
example,  19.3  per  cent  of  applications  (i.e. 
268/1387  in  Table  1)  for  project  grants  in 


1996  were  from  women,  compared  to  18.9  per 
cent  of  awarded  grants  and  19.5  per  cent  of 
failed  grants.  In  other  words,  there  is  no  evid- 
ence that  applications  for  project  grants  from 
women  are  more,  or  less,  likely  to  fail  than 
those  from  men. 

The  age  and  sex  distributions  of  project  grant 
applicants  are  graphically  represented  in  the 
population  pyramid'  in  Figure  1.  The  female 
population  is  on  the  left-hand  side  and  the 
male  population  on  the  right,  whilst  the  young 
are  at  the  bottom  and  the  old  are  at  the  top.5  It 
is  striking  from  Figure  1  that  four  times  as 
many  men  apply,  and  are  awarded,  project 
grants  as  women.  That  is,  for  every  100  male 
applicants  there  are  only  24  female  applicants. 
The  figure  also  shows  that,  whilst  the  sex  ratio 
(the  number  of  men  divided  by  the  number  of 
women)  broadly  increases  with  age,  the  success 
rate  for  men  and  women  is  roughly  the  same 
for  all  but  the  youngest  and  two  oldest  age 
bands  (where  small  numbers  make  the  results 
inconclusive). 


Table  1  The  outcome  of  project  grant  applications 


Number  of  applications 
Number  of  successful  applications 
Success  rate  (%) 


Men 

1097 
302 
27.5 


Women 


Unsexed 


268 
72 
26.9 


22 
7 
31.8 


Total 

1387 
381 

27.5 


a  The  expenditure  figure 
excludes  a  £60m  grant  for  the 
Sanger  Centre,  a  genome 
research  centre  set  up  in 
1992  by  the  Wellcome  Trust 
and  the  Medical  Research 
Council. 

b  The  sex  of  applicants  was 
determined  from  their  first 
name.  Where  recorders  were 
unfamiliar  with  the  name,  it 
was  noted  and  looked  up  on 
the  Internet  site, 
www.thebabynet.com. 
'Unisex'  names  were  shown 
to  the  Trust  staff,  and  were 
classified  only  if  the 
applicant's  sex  was  known. 
The  remaining  names  were 
classified  as  'unsexed'. 

c  It  should  be  noted  that  some 
project  grant  applications 
were  missing  as  some  files 
would  have  been  on  loan 
from  the  registry  and 
omissions  may  have  been 
made  by  the  recorders.  As 
long  as  there  was  not  a 
systematic  sex  bias  in  the 
missing  grant  applications 
then  the  following  results  will 
not  be  compromised. 
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Figure  1:  Numbers  of  rejected  and  awarded  project  grant  applications  by  age  and  sex 


1  Throughout  this  report  the 
z-test  is  used  to  test 
differences  between  the 
bibliometric  indices.6 


To  examine  whether  women  need  to  have  a 
better  publication  record  than  men  to  win 
grants,  the  scientific  papers  authored  by  the 
applicants  were  collated  from  bibliographic 
databases.  Detailed  methods  and  results  of 
these  analyses  are  presented  in  the  Annexes 
(pp.  18—20).  The  publication  records  of  a  sub- 
sample  of  25  male  and  25  female  project  grant 
applicants  are  summarized  in  Figure  2  and 
illustrate  that  men  and  women  have  nearly 
identical  publication  patterns.  For  example, 
the  first  graph  (Figure  2a)  shows  that  the 
average  number  of  papers  published  a  year 
by  male  and  female  project  grant  applicants 
to  the  Trust  is  very  similar  (an  average  of  2.5 
papers  a  year). 

However,  in  as  much  as  publishing  a  paper 
is  an  indication  of  success,  some  researchers 
will  present  their  findings  in  higher-impact 


journals  such  as  Nature  and  Science,  and  others 
will  submit  papers  to  less  well-known  titles. 
The  second  graph  (Figure  2b)  illustrates  the 
expected  number  of  citations  a  paper  would 
receive  in  the  five  years  following  publication, 
on  the  basis  of  the  journal  in  which  the  papers 
were  published.  This  indicator  demonstrates 
that  women  publish  in  journals  of  a  marginally 
higher  impact  than  men,  although  this  result  is 
not  statistically  significant  (i.e.  p  >  0.05).  The 
final  indicator  presented  in  Figure  2c  classifies 
the  research  type  of  the  journal  in  which  the 
grant  applicants'  papers  appear.  This  is  on  a 
scale  of  1  for  clinical  observation  to  4  for  basic 
research,  as  indicated  by  the  journal  in  which 
the  papers  are  published.  Consequently,  Figure 
2c  shows  that  although  women  work  in 
slightly  more  clinical  fields  than  men,  both 
groups  are  doing  very  basic  work. 
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fHE  OUTCOME  OF  WELLCOME  TRUST  GRANT  APPLICATIONS 


Figure  2:  Publication  patterns  for  a  sample  of  successful  project  grant  applications 


Clinical 
observation 


Basic 
research 


c:  Research  type 


2.2  PROGRAMME  GRANTS, 
1994-1996 


Programme  grants  are  similar  to  project  grants 
but  are  usually  for  larger  amounts  of  money  and 
are  awarded  for  typically  over  five  years  in  the 
first  instance.4  In  1995/96,  15  per  cent  of  the 
Trust's  research  expenditure  was  on  programme 
grants,  although  they  only  made  up  3  per  cent 
of  the  awards  by  number  that  academic  year. 

Applications  for  programme  grants  go  through 
a  number  of  steps  before  an  award  is  made.  In 
all  cases  there  is  a  preliminary  application  stage 


prior  to  invitation  to  submit  a  full  formal  grant 
application,  which  is  then  subject  to  the  normal 
competitive  evaluation  and  peer  review.  It  is  the 
outcomes  of  formal  applications  for  programme 
grants  accepted  between  1994  and  1996  which 
we  have  examined  in  this  report.  As  a  result  of 
the  preliminary  screening  process  the  overall 
success  rate  was  quite  high  (50.4  per  cent) 
although,  as  shown  in  Table  2,  women  were 
more  successful  than  men  (62.2  per  cent  for 
women  versus  47.8  per  cent  for  men). 
Nevertheless,  it  is  also  evident  that  five  times  as 
many  men  applied  to  the  Trust  for  programme 
grants  than  women. 
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Table  2  The  outcome  of  full  programme  grant  applications 


Men  Women  Unsexed  Total 


Number  of  applications  115  23  1  139 

Number  of  successful  applications  55  15  0  70 

Success  rate  (%)  47.8  62.2  ~  50.4 


Figure  3:  Publication  patterns  for  successful  programme  grant  applications 


b:  Expected  impact 
(average  citations 
over  five  years/ 
paper) 


c:  Research  type 


A 


The  publication  patterns  for  successful  pro- 
gramme grant  applications  are  summarized  in 
Figure  3  (and  are  presented  in  more  detail  in  the 
Annexes).  These  results  show  that  successful 
male  programme  grant  applicants  publish 


significantly  (i.e.  p  <  0.05)  more  papers  per  year 
than  their  female  colleagues,  but  there  is  no 
difference  in  the  impact  of  the  journals  in  which 
they  publish  (Figure  3b). 


Women  and  Peer  Review:  an  audit  of  the  wellcome  trust's  decision-making  on  grants 


9 


RANT  APPLICATIONS 


SENIOR  RESEARCH  FELLOWSHIPS  IN 
BASIC  BIOMEDICAL  SCIENCE, 
1994/95-1996/97 


The  Senior  Research  Fellowships  in  Basic 
Biomedical  Science  (SBBF)  is  a  scheme  that 
enables  scientifically  qualified  workers  of 
exceptional  ability  to  develop  their  programme 
of  research.4  Candidates  are  young  investig- 
ators who  have  shown  special  promise  in  their 
initial  studies  of  basic  biomedical  problems, 
normally  with  between  five  and  ten  years' 
research  experience  from  the  date  of  their  doc- 
toral degree.  All  applications  made  between 
1994/95  and  1996/97  for  SBBF  were  exam- 
ined/ Awards  are  made  in  an  annual  competi- 
tion, commencing  with  advertisements  in  the 
scientific  media  for  preliminary  applications. 
For  the  three  academic  years  studied,  there 
were  354  preliminary  applications  (Figure  4). 
The  applications  are  each  assessed  by  three 
members  of  the  Basic  Science  Interest  Group 
(BSIG),  which  invites  candidates  to  submit  a 
full  application.  The  returned  applications 
(with  referees'  reports)  are  then  scored  by  all 
members  of  the  BSIG  and,  on  the  basis  of  an 


average  score,  a  short-list  for  interview  is  drawn 
up.  Two  days  of  interviews  are  held  each  year, 
and  at  this  stage  some  candidates  may  be 
awarded  a  Research  Career  Development 
Fellowship  (RCDF). 

Figure  4  summarizes  the  outcome  of  applica- 
tions to  the  Trust  for  SBBF.  From  the  354  pre- 
liminary applications,  21  SBBF  and  seven 
RCDF  were  awarded.  The  overall  success  rate 
for  SBBF  was  5.9  per  cent.  Female  applicants 
were  more  successful  than  males  (8.6  per  cent 
versus  5.5  per  cent).  This  is  reflected  in  the 
declining  sex  ratio  of  male  to  female  applicants 
through  the  decision-making  process  (Figure 
4).  For  example,  although  twice  as  many  men 
as  women  initially  applied  to  the  Trust  for  an 
SBBF,  only  a  third  more  were  awarded  full  fel- 
lowships. Conversely,  six  times  as  many  men  as 
women  were  awarded  the  intermediate  RCDF 
(an  award  designed  to  help  postdoctoral  in- 
vestigators develop  an  independent  research 
career  in  basic  biomedical  science).4  There  is 
thus  no  evidence  of  discrimination  against 
women  in  the  Trust's  decision-making  process 
in  awarding  SBBF. 


'A  small  number  of  candidates 
linked  to  major  initiatives  are 
automatically  entered  at  the 
full  application  stage.  These 
candidates  have  been  omitted 
from  this  anal/sis. 
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Figure  4:  The  outcome  of  SBBF  applications 


354  preliminary  applications 
(220M  :  105F  :  29U) 
M  :  F  =  2.1  :  1 


7  RCDF 
(6M  :  1 F) 
M  :  F  =  6.0  :  1 


123  full  applications 
invited 

(86M  :  35F  :  2U) 
M  :  F  =  2.5  :  1 


59  interviewed 
(39M  :  20F) 
M  :  F  =  1.9  :  1 


31  rejected 
(21 M  :  10F) 
M  :  F  =  2.1  :  1 


21  SBBF 
(12M  :  9F) 
M  :  F  =  1.3  :  1 


M  =  male;  F  =  female;  U  =  unsexed 

RCDF  =  Research  Career  Development  Fellowships 

SBBF  =  Senior  Research  Fellowships  in  Basic  Biomedical  Science 


The  main  reason  for  analysing  the  outcome  of 
SBBF  was  that  the  applicants  are  scored  by  the 
BSIG  in  order  to  determine  whom  to  invite  for 
interview.  This  allowed  us  to  compare  the 
scores  awarded  to  male  and  female  applicants. 
The  mean  score  for  the  interviewees  was  7.49 
(out  of  10),  compared  to  5.45  for  those  people 


who  submitted  full  applications,  but  were  not 
interviewed.  Table  3  presents  the  mean  scores 
by  outcome  of  interview  and  sex  of  applicant. 
In  this  table  there  is  no  statistically  significant 
(i.e.  p  >  0.05)  difference  between  the  scores  of 
the  male  and  female  candidates. 
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THE  OUTCOME  OF  WELLCOME  TRUST  GRANT  APPLICATIONS 


Table  3  SBBF  score  for  interviewed  applicants  by  outcome  and  sex* 


Rejected 


Men  Mean  7.29 

n  21 
SE  0.12 

Women  Mean  7.21 

n  10 
SE  0.19 

Total  Mean  7.27 

n  31 
SE  0.10 


RCDF  SBBF  Total 

7.31  8.09  7.53 

6  12  39 
0.20  0.24  0.11 
6.90  7.65  7.39 
1  9  20 

0.21  0.14 

7.25  7.90  7.49 

7  21  59 
0.18  0.15  0.10 


*The  number  of  applications  is  represented  by  n  and  the  standard  error  by  SE. 

RCDF  =  Research  Career  Development  Fellowships 

SBBF  =  Senior  Research  Fellowships  in  Basic  Biomedical  Science 


Bibliometric  analyses  of  all  applicants  who 
were  interviewed  for  SBBF  (n  =  59;  Figure  4) 
are  presented  in  Figure  5.  The  only  statistically 
significant  (i.e.  p  <  0.05)  result  is  the  lower 
journal  impact  factor  for  women,  when  com- 


pared to  men  (Figure  5b).  Therefore,  as  with 
programme  grants,  this  analysis  suggests  that 
men  need  to  have  a  better  track  record  than 
women  to  be  interviewed  for  an  SBBF. 


Figure  5:  Publication  patterns  of  applicants  who  were  interviewed  for  SBBF 


Clinical 
observation 


Basic 
research 


c:  Research  type 
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DISCUSSION 


3. 1  AGE  AND  SEX  CHARACTERISTICS 
OF  THE  BIOMEDICAL  COMMUNITY 

We  conclude  from  the  preceding  analysis  that 
there  is  no  evidence  of  discrimination  against 
women  in  the  Trust  s  decision  making  on  grant 
applications.  The  award  rates  are  about  the 
same  for  women  and  men  in  the  three  schemes 
analysed,  and  the  publication  records  of  suc- 
cessful applicants  are  also  similar. 

It  is  clear,  however,  that  more  men  apply  to  the 
Trust  for  research  grants  than  women.  There 
are  four  times  as  many  male  as  female  applic- 
ants for  project  grants  (Table  1),  five  times  as 
many  for  programme  grants  (Table  2),  and 
twice  as  many  for  SBBF  (Figure  4).  One  ex- 
planation for  these  differences  would  be  that 
there  are  more  men  working  in  biomedical 
science  than  women  in  the  UK.  This  is  not  the 
case,  as  is  evident  in  the  population  pyramid  in 
Figure  6,  which  describes  the  characteristics  of 
all  biomedical  scientists  working  in  UK 
universities.  There  were  only  marginally  more 
male  academic  biomedical  staff  than  female  in 
1995/96,  and  the  distribution  of  staff  within 
the  age  bands  is  almost  even.  In  the  under-25 
age  group  there  were  one-and-a-half  times 
more  female  biomedical  academics  than  male. 


'These  data  were 
commissioned  from  the 
Higher  Education  Statistics 
Agency  (HESA).  Biomedical 
science  is  defined  as  all  full- 
time  academic  staff  working 
in  the  following  cost  centres: 
clinical  medicine,  clinical 
dentistry,  veterinary  science, 
anatomy  and  physiology, 
nursing  and  paramedical 
studies,  health  and 
community  studies, 
psychology  and  behavioural 
sciences,  pharmacy, 
pharmacology  and 
biosciences. 


Consequently  it  seems  that  women  do  not 
apply  to  the  Trust  for  grants  in  the  numbers 
that  would  be  expected.  In  1994  the  Trust 
required  applicants  to  hold  tenured  university 
positions;  a  policy  that  would  have  restricted 
the  number  of  female  applicants  as,  in 
1995/96,  1.59  times  more  men  than  women 
held  tenured  positions  in  universities.7 
However,  current  Trust  policy  permits  applica- 
tions from  individuals  without  tenured  posi- 
tions, as  long  as  they  have  a  sponsor  in  the 
same  department  who  does  have  an  established 


post  and  can  guarantee  space,  facilities  etc.  As 
a  result  of  this  change  in  policy,  for  every  100 
men  who  are  eligible  to  apply  to  the  Trust 
there  are  78  women  that  are  also  eligible  (Table 
4  and  Figure  6).  In  fact,  we  found  that  for 
every  100  male  applicants  for  project  grants 
there  were  only  24  females  who  applied  for  the 
same  award  (Table  4  and  Figure  1). 

Applicants  for  programme  grants  and  SBBF 
will,  on  the  whole,  be  more  senior  academics. 
However,  at  the  senior  grades  (i.e.  senior  lec- 
turers, readers  and  professors)  there  is  an  even 
greater  bias  towards  men;  for  every  100  eligible 
men,  there  are  28  eligible  women  which, 
although  still  more  than  the  20  observed  pro- 
gramme grant  applications,  is  a  smaller  differ- 
ence than  for  project  grants  (Table  4). 
Interestingly,  for  SBBF,  for  every  100  male 
candidates  there  were  48  female  candidates.  It 
is  difficult  to  estimate  a  comparator  popula- 
tion for  this  group,  as  applications  will  occur 
five  to  ten  years  after  the  award  of  a  doctoral 
degree.  However  it  is  likely  that  the  expected 
number  of  female  applicants  for  SBBF  will  be 
between  the  numbers  expected  for  project  and 
programme  grants  (i.e.  between  28  and  78 
respectively),  which  is  very  similar  to  the 
observed  number  (Table  4). 

In  other  words,  it  seems  that  women  do  not 
apply  to  the  Trust  for  project  or  programme 
grants  in  the  numbers  that  would  be  expected, 
but  they  do  for  SBBF.  It  is  difficult  to  draw 
conclusive  inferences  from  these  results,  except 
to  note  the  SBBF  is  a  career  development 
award,  whilst  the  other  two  grants  support 
project-orientated'  research.  This  may  indic- 
ate that  women  prefer  fellowship  schemes,  but 
such  an  observation  merits  further  research. 
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Table  4    Comparison  of  observed  and  expected  number  of  female  applicants 
per  1 00  male  applicants 


Scheme                                   Expected  number  Observed  number 

of  female  applicants  of  female  applicants 

Project  grants                                        78  24 

Programme  grants                                  28  20 
Senior  Research  Fellowships  in 

Basic  Biomedical  Science                          28-78  48 


14 


Women  and  Peer  Review:  an  audit  of  the  wellcome  trust's  decision-making  on  grants 


Figure  7:  Proportions  of  academic  staff  according  to  discipline,  status  and  sex,  1995/967 
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Finally  it  is  worth  noting  that  the  dearth  of  eli- 
gible female  academics  at  the  senior  academic 
grades  is  further  accentuated  at  professorial 
level.  For  example,  within  biomedical  sciences 
only  one  in  nine  professors  is  a  woman  (Figure 
7).  This  may  either  reflect  past  employment 
patterns  or  be  the  product  of  systematic  biases 
against  women  in  academia.  Accordingly,  only 
in  20-25  years'  time,  when  today's  junior  staff 
are  being  appointed  as  professors,  will  it  be 
possible     to     examine     the     extent  of 


discrimination  in  today's  biomedical  institu- 
tions. This  is  illustrated  in  Figure  8,  where  the 
proportion  of  women  in  biomedical  science 
has  changed  considerably  over  the  past  decade 
(the  overall  sex  ratio  has  halved  from  2.90  in 
1987/88  to  1.28  in  1995/96).  As  women  are 
now  better  represented  in  the  non-professorial 
grades,  it  can  be  anticipated  that  there  will  be  a 
greater  proportion  of  female  professors  in  the 
future,  unless  institutional  discrimination 
is  occurring. 
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•  DISCUSSION 


Figure  8:  Biomedical  academic  staff  by  sex  and  grade,  1987/88,  and  1995/96  7'' 


CONCLUSION  AND  POLICY 
IMPLICATIONS 

The  study  has  demonstrated  that  there  is  no 
discrimination  against  women  in  the  assess- 
ment of  applications  for  Wellcome  Trust 
project  grants,  programme  grants  or 
SBBF.  Nevertheless,  it  seems  that  there  are 
more  subtle  barriers  inhibiting  women  from 
applying  to  the  Trust  than  eligibility.  It  is 
significant  that  not  as  many  women  apply  to 


the  Trust  as  would  be  anticipated,  and  it  is 
important  that  the  reason(s)  for  this  are  exam- 
ined. We  do  not  know  if  this  is  specific  to  the 
Trust,  or  is  a  pattern  experienced  by  other  bio- 
medical (or  non-biomedical)  research  funding 
agencies.  We  therefore  recommend  that  funding 
bodies  should,  as  part  of  their  efforts  to  ensure 
equal  opportunities,  work  together  to  invest- 
igate the  factors  that  prevent  women  from 
applying  for  grants  in  the  numbers  expected. 


16 


Women  and  Peer  Review:  an  audit  of  the  wellcome  trust's  decision-making  on  grants 


REFERENCES 


1  HMSO  (1994)  The  Rising  Tide.  A  report  on 
women  in  science,  engineering  and 
technology.  HMSO,  London 

2  O'DriscollM,  Anderson  J  (1994)  Women 
in  Science.  Attitudes  of  university  students 
towards  a  career  in  research:  a  pilot  study. 
PRISM  Report  No.  4.  The  Wellcome 
Trust,  London  (ISBN:  1  869835  417) 

3  Wenneras  C,  Wold  A  (1997)  Nepotism 
and  sexism  in  peer-review.  Nature  387: 
341-343 

4  The  Wellcome  Trust  (1997)  Grants  and 
Support  for  Biomedical  Research.  The 
Wellcome  Trust,  London 

5  Newell  (1988)  Methods  and  Models  in 
Demography.  Belhaven  Press,  London 

6  Kirkwood  (1988)  Essentials  of  Medical 
Statistics.  Blackwell  Science,  Oxford 

7  HESA  (1996)  Resources  of  Higher 
Education  Institutions,  199415.  Higher 
Education  Statistics  Agency,  Cheltenham 

8  USR  (1988)  University  Statistics  Records, 
198718.  University  Statistics  Records, 
Cheltenham 

9  Narin,  Pinski,  Gee  (1976)  Structure  of 
the  biomedical  literature.  Journal  of  the 
American  Society  for  Information  Science 
27  (1):  25-45 

10  ISI  (1990)  Journal  Expected  Citations 
Rates  File.  Institute  of  Scientific 
Information,  Philadelphia 

11  Lewison  G  (1996)  The  definition  of 
biomedical  research  subfields  with  title 
keywords  and  application  to  the  analysis 
of  research  outputs.  Research  Evaluation 
6(1):  25-36 


Women  and  Peer  Review:  an  audit  of  the  wellcome  trust's  decision-making  on  grants 


17 


B I BLIO METRIC  ANALYSIS 


METHODS 

The  bibliometric  analysis  of  Trust  applicants  was  based  on  searches  of  the  Science  Citation  Index  (SCI)  and 
Social  Sciences  Citation  Index  (SSCI)  for  the  five  years  preceding  the  application  to  the  Trust.  Searches  were 
conducted  on  the  applicant's  name  (e.g.  SMITH-JB)  and  bibliographic  details  of  the  papers  identified  were 
downloaded.  Care  was  taken  to  include  women  whose  name  may  have  changed  following  marriage,  by 
referring  to  the  bibliographies  sent  in  by  applicants.  The  data  were  analysed  on  an  Excel  spreadsheet,  and  for 
each  candidate  the  papers  were  filtered  by  first  his/her  name  and  then  by  their  past  three  academic 
addresses.  Once  applicants'  papers  were  identified,  a  series  of  different  bibliometric  indices  were  calculated: 

Count  of  papers  (N) 

The  total  number  of  papers  retrieved  from  the  SCI  and  SSCI  over  a  five-year  period  were  limited  to 
articles,  notes  and  reviews  in  accordance  with  normal  bibliometric  analysis  of  substantive  research 
outputs.  In  Figures  2,  3  and  5  this  indicator  is  presented  as  the  number  of  papers  per  year. 

Mean  research  type  (RT) 

To  characterize  the  research  outputs  of  the  applicants,  the  journals  in  which  the  papers  were  published 
were  classified  into  four  categories  (1  =  clinical  observation;  2  =  clinical  mix;  3  =  clinical  investigation;  and 
4  =  basic  research).  The  values  of  RT  for  each  journal  have  been  allocated  on  the  basis  of  inspection  of  the 
journal  by  experts  and  the  citation  pattern  of  the  papers  within  it.9 

Expected  impact  (C0  4) 

Five-year  citation  records,  taken  as  the  average  number  of  citations  received  by  items  published  in  1990 
and  cited  in  journals  processed  for  the  SCI/SSCI  in  the  years  1990-94. 10 

Mean  journal  weighting  (W) 

The  journals  in  which  the  papers  were  published  were  classified  into  four  categories  on  the  basis  of  their 
influence  or  weighting  (W),11  as  determined  by  their  mean  CQ  4  value.  For  journals  with  a  C0  4  value  >  20, 
W  =  4;  for  C(M<  20  but  >  10,  W  =  3;  for  CQ  4  <  10  but  >  5,  W  =  2;  and,  for  C0  4  <  5,  W  =  1. 

Mean  number  of  authors  (A) 

The  mean  number  of  authors  on  a  paper. 

Total  influence  per  author  (I) 

The  sum  of  the  ratio  of  the  journal  weighting  (W)  to  the  number  of  authors  (A)  for  each  paper  published 
by  the  applicant. 
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ANNEX  2 


BIBLIOMETRIC  ANALYSIS 

RESULTS 
Project  grants 

In  order  to  restrict  the  workload  to  a  manageable  number  of  searches  on  the  SCI  and  SSCI,  the  biblio- 
metric  analysis  of  successful  project  grant  applicants  was  restricted  to  a  random  sample  of  25  men  and  25 
women.  The  results  of  this  analysis  are  presented  in  Table  A1 ,  and  demonstrate  that  there  were  no 
statistically  significant  differences  (i.e.  p  >  0.05)  between  women  and  men  for  the  various  bibliometric  indices. 

Programme  grants 

Bibliometric  analyses  for  all  successful  applications  for  programme  grants  are  shown  in  Table  A2,  and 
demonstrate  that  there  are  statistically  significant  sex  differences  (i.e.  p  <  0.05)  in  publication  records  for 
the  number  of  publications  (N),  and  their  influence  (I). 

SBBF 

Bibliometric  analyses  of  all  applicants  who  were  interviewed  for  SBBF  (n  =  59;  Figure  4)  are  presented  in 
Table  A3.  The  only  statistically  significant  result  (i.e.  p  <  0.05)  is  for  the  expected  impact  (C0  .)  for  women, 
when  compared  to  men. 
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Table  A1  Bibliometric  analysis  of  a  sample  of  successful  project  grant  applications 


N 

RT 

c0.4 

W 

A 

1 

Females 

Means 

11.20 

3.37 

13.27 

2.44 

4.06 

8.36 

SE 

1.66 

0.14 

1.75 

0.13 

0.27 

1.16 

Males 

Means 

13.80 

3.47 

11.75 

2.22 

3.69 

10.54 

SE 

1.64 

0.11 

1.13 

0.09 

0.25 

0.99 

Total 

Means 

12.50 

3.42 

12.51 

2.33 

3.88 

9.45 

SE 

1.65 

0.12 

1.47 

0.11 

0.26 

1.09 

Table  A2  Bibliometric  analysis  for  successful  programme  grant  applications^ 


N  RT  C04  W  A  I 


Females 

Means 

13.78* 

3.50 

18.27 

2.73 

3.89 

11.57* 

SE 

2.38 

0.16 

1.96 

0.16 

0.33 

1.84 

Males 

Means 

20.52 

3.67 

21.15 

2.94 

4.18 

17.97 

SE 

1.94 

0.06 

1.55 

0.06 

0.18 

1.54 

Total 

Means 

19.38 

3.64 

20.66 

2.90 

4.13 

16.88 

SE 

1.71 

0.06 

1.34 

0.06 

0.16 

1.37 

*  Indicates  a  significant  difference  between  women  and  men  (i.e.  p  <  0.05). 

'A  number  of  candidates  (i.e.  five  women  and  12  men)  were  omitted  from  this  analysis  as  their  files  were  not  in  the 
registry.  This  could  have  a  distorting  effect  on  the  results  presented  in  this  table  especially  given  the  low  number  of 
women  (n  =  15;  Table  2). 


Table  A3  Bibliometric  analysis  of  applicants  who  were  interviewed  for  SBBF 


N  RT  C04  W  A  I 


Females 

Means 

11.80 

3.53 

22.72* 

2.80 

4.18 

10.08 

SE 

1.12 

0.14 

2.82 

0.13 

0.40 

1.14 

Males 

Means 

14.31 

3.62 

29.26 

3.09 

4.69 

12.03 

SE 

1.49 

0.06 

2.36 

0.08 

0.25 

1.33 

Total 

Means 

13.46 

3.59 

27.04 

2.99 

4.52 

11.37 

SE 

1.06 

0.06 

1.86 

0.07 

0.22 

0.96 

*  Indicates  a  significant  difference  between  women  and  men  (i.e.  p  <  0.05). 
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